Prompt Generator for Stable Diffusion#

Note

Install ekorpkit package first.

Set logging level to Warning, if you don’t want to see verbose logging.

If you run this notebook in Colab, set Hardware accelerator to GPU.

!pip install ekorpkit[art] exit()

Preparing the environment#

%config InlineBackend.figure_format='retina'
%load_ext autotime

from ekorpkit import eKonf

eKonf.setLogger("WARNING")
eKonf.set_cuda(device=0)
print("version:", eKonf.__version__)

is_colab = eKonf.is_colab()
print("is colab?", is_colab)
if is_colab:
    eKonf.mount_google_drive()
project_dir = eKonf.set_workspace(workspace="/content/drive/MyDrive/workspace/", project="ekorpkit-book")
print("project_dir:", project_dir)

INFO:ekorpkit.base:Setting cuda device to ['A100-SXM4-40GB (id:0)']
INFO:ekorpkit.base:Setting EKORPKIT_WORKSPACE_ROOT to /content/drive/MyDrive/workspace/
INFO:ekorpkit.base:Setting EKORPKIT_PROJECT to ekorpkit-book
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env

The autotime extension is already loaded. To reload it, use:
  %reload_ext autotime
version: 0.1.40.post0.dev8
is colab? False
project_dir: /content/drive/MyDrive/workspace/projects/ekorpkit-book
time: 28.6 ms (started: 2022-11-09 04:41:22 +00:00)

Load a Generator and Generate Prompts#

from ekorpkit.models.art.prompt import PromptGenerator
from ekorpkit.models.art.stable import StableDiffusion

pgen = PromptGenerator(verbose=False)
sd = StableDiffusion()

2022-11-09 03:42:24.640386: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env

time: 7.12 s (started: 2022-11-09 03:42:23 +00:00)

text_prompts = pgen.generate_prompts(prompt="people looking out the lonely city street from windows of buildings in a Edward Hopper style")
text_prompts

['people looking out the lonely city street from windows of buildings in a Edward Hopper style. detailed, romantic, enchanting, trending on artstation.',
 'people looking out the lonely city street from windows of buildings in a Edward Hopper style, painted by Hans Fredrik Gude, Earl Moran, William-Adolphe Bouguereaume, trending on artstation',
 'people looking out the lonely city street from windows of buildings in a Edward Hopper style. Highly detailed digital painting, artstation, concept art, smooth, sharp foccus ilustration, Artstation HQ.',
 'people looking out the lonely city street from windows of buildings in a Edward Hopper style, painting by Ivan Bilibin, highly detailed, intricate, smooth, sharp focus, trending on ArtStation, concept art, illustration, art by Krenz',
 'people looking out the lonely city street from windows of buildings in a Edward Hopper style, macro lens, shallow depth of field, highly detailed, digital painting, trending artstation, concept art, illustration, cinematic lighting, vibrant colors, photorealism',
 'people looking out the lonely city street from windows of buildings in a Edward Hopper style by Edward Hopper, painted by James Gilleard, surrealism, airbrush, Ilya Kuvshinov, WLOP, Stanley Artger',
 'people looking out the lonely city street from windows of buildings in a Edward Hopper style. highly detailed science fiction painting by norman rockwell, frank frazetta, and syd mead. rich colors, high contrast, gloomy atmosphere, dark',
 'people looking out the lonely city street from windows of buildings in a Edward Hopper style, painting by Michelangeloangelo and Norman Rockwell, highly detailed, excellent composition, cinematic concept art, dramatic lighting, trending on artstation',
 'people looking out the lonely city street from windows of buildings in a Edward Hopper style, surrealism, sssociety, artstation hd, winner, weirdcore, photorealism, high definition, insane details, intricate details, h',
 'people looking out the lonely city street from windows of buildings in a Edward Hopper style, detailed portraits, nostalgic atmosphere, volumetric lighting, 8k, Artstation, concept art, soft light, hdri, smooth, sharp focus, illustration']

time: 1.15 s (started: 2022-11-09 04:18:09 +00:00)

Batch Run#

batch_name = "batch-run-test"
random_seed = False
batch_run_params = {
    "text_prompts": text_prompts[:4],
    "guidance_scale": [5, 10],
    "num_inference_steps": [50,100],
}
batch_run_pairs = [
#     ("guidance_scale", "num_inference_steps"),
    ['text_prompts']
]
batch_results = sd.batch_imagine(
    batch_name=batch_name, 
    batch_run_params=batch_run_params, 
    batch_run_pairs=batch_run_pairs,
    random_seed=random_seed,
    seed=123,
    text_prompts="Brave and beautiful new world in Jeju, artstation matte painting.",
    num_samples=3,
    max_display_image_width=800,
)
# print(batch_results)

Prompt: people looking out the lonely city street from windows of buildings in a Edward Hopper style. detailed, romantic, enchanting, trending on artstation.
../../../_images/prompt-generator_9_1.png 
Prompt: people looking out the lonely city street from windows of buildings in a Edward Hopper style, painted by Hans Fredrik Gude, Earl Moran, William-Adolphe Bouguereaume, trending on artstation
../../../_images/prompt-generator_9_3.png 
Prompt: people looking out the lonely city street from windows of buildings in a Edward Hopper style. Highly detailed digital painting, artstation, concept art, smooth, sharp foccus ilustration, Artstation HQ.
../../../_images/prompt-generator_9_5.png 
Prompt: people looking out the lonely city street from windows of buildings in a Edward Hopper style, painting by Ivan Bilibin, highly detailed, intricate, smooth, sharp focus, trending on ArtStation, concept art, illustration, art by Krenz
../../../_images/prompt-generator_9_7.png 
time: 2min 7s (started: 2022-11-09 04:18:11 +00:00)

# run_config_path = "/workspace/projects/ekorpkit-book/.MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/stable-diffusion/batch-run-test/configs/batch-run-test_guidance_scale_num_inference_steps(0)_run_configs.yaml"
# results = sd.batch_collage(run_config_path, max_display_image_width=800)

Train a Generator#

pgen.train(model_name="ekorpkit/stable-prompts-2")

pgen.load_model(model_name="ekorpkit/stable-prompts-2")